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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1 .1 14, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 16 
January 2007 has been entered. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and- 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-14, 16-25, 28, and 29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Brown et al (6,868,225) in further view of Newman et al (6,154,600) 

Regarding claim 1, Brown et al disclose an audiovisual system comprising: 

• a system controller (Fig. 1 , CPU 106); 

• a storage device to store the portion of the audiovisual data and to play back 
the stored portion of the audiovisual data (Fig. 1, Hard Disk 105); 

• a marking module to create metadata in response to a control input for 
marking the programs segments comprising information regarding the 
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program segments of the stored portion of the audiovisual data (Col 3, lines 
58-59 "Bookmarks are stored [and] retrieved for the user"); 

• a display generator to generate a mosaic representation of the program 
segments of the stored portion of the audiovisual data (Col 15, lines 43-44 
"The invention places an indicator 1808 on the screen, indicating that a 
bookmark has been saved" and Col 20, lines 8-9 "index or bookmark 
indicators are displayed over the trick play bar"); 

• wherein the mosaic representation comprises a plurality of cells representing 
respective stored program segments of a program (Col 14, lines 44-47 "The 
Now Showing screen 1701 is shown which has a list of programs 1705 that 
are displayed to the user in descending order with the most recently obtained 
program at the top of the list"); and 

• a program selector to select a program segment of the stored portion of the 
audiovisual data in response to a user input based on the information of the 
metadata, enabling the user to jump to and play back selected program 
segments of the plurality of programs (Col 15, lines 44-45 "The user can, at 
any time, access any of his bookmarks and continue viewing the program 
from the bookmark"). 

Brown discloses images providing the user with additional data regarding 
the recorded program segments of a program (Col 16, lines 53-62), but does not 
explicitly disclose those images as being extracted from the respective stored 
program segments. 
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Newman teaches an audiovisual recording and reproducing system 
comprising a menu of program segments comprising images extracted from the 
respective stored program segments (Col 15, lines 4-8 "the storyboard GUI 440 
includes a shot tab 404 displaying icons 406 representing previously captured 
clips. The storyboard GUI 440 likewise includes a display window 408 to display 
an image from a clip referenced by a selected icon 410"). 

As taught by Newman, a representation of a video program segment using 
a representative image extracted from the video program segment is well known, 
providing the user with a visual reference to the visual elements of the segment 
and simplifying the user's process of deciding which segment to view. 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention to modify Brown in order to include a representative 
image extracted from the video segment in an index of those video segments. 
Regarding claim 2, Brown et al disclose an audiovisual system comprising a 

personal video recorder (TiVo, the assignee, is known as a manufacturer of PVRs). 
Regarding claim 3, Brown et al disclose an audiovisual system wherein the 

storage device comprises a read/write, random-access, non-volatile storage media (Fig 

1, Hard Disk 105). 

Regarding claim 4, Brown et al disclose an audiovisual system wherein the 
metadata comprises user-generated modifications to pre-existing metadata (Col 5, lines 
47-52 "when the parser 401 sees a private data event, it directs the byte stream to the 
private data DMA engine 404 and directs an event to the event buffer 413. The Media 
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Switch notifies the program logic via an interrupt mechanism when events are placed in 
the event buffer" and Col 1 , lines 53-55 "The system allows a user to save and retrieve 
bookmarks for several audio and/or video programs on a single device"). 

Regarding claim 5, Brown et al disclose an audiovisual system wherein the pre- 
existing metadata is received by the audiovisual system with the audiovisual data (Col 
6, lines 65-67 "The demultiplexer multiplexes the extracted audio, video and private 
data channel streams through the video input Media Switch port"). 

Regarding claim 6, Brown et al disclose an audiovisual system wherein the 
metadata is stored subsequently to the storing of the stored portion of the audiovisual 
data by the audiovisual system (Col 1 , lines 58-60 "a bookmark function that allows the 
user to bookmark a program where he left off'). 

Regarding claim 7, Brown et al disclose an audiovisual system comprising a 
preference determination module, and wherein the control input comprises signals from 
the preference determination module (Col 14, lines 50-52 "The system's list is based 
upon the program preferences that the user has expressed to the system using the 
thumbs up and thumbs down ratings"). 

Regarding claim 8, Brown et al disclose an audiovisual system wherein the 
metadata comprises information indicating a level of interest by a user in the program 
segment starting from the corresponding program location (Col 14, lines 50-52 "The 
system's list is based upon the program preferences that the user has expressed to the 
system using the thumbs up and thumbs down ratings"). 



Application/Control Number: 09/877,632 Page 6 

Art Unit: 2621 

Regarding claim 9, Brown et al disclose an audiovisual system wherein the 
metadata comprises information indicating a category of the program segment starting 
from the corresponding program location (Col 23, lines 20-22 "The program guide 
information is sorted in different ways, e.g., all movies 2901, favorite channels 3001, all 
channels 3101, all sports, etc."). 

Regarding claim 10, Brown et al disclose an audiovisual system wherein the 
category comprises the genre of the program segment starting from the corresponding 
program location (Col 23, lines 20-22 'The program guide information is sorted in 
different ways, e.g., all movies 2901 , favorite channels 3001, all channels 3101, all 
sports, etc."). 

Regarding claim 1 1 , Brown et al disclose an audiovisual system wherein the 
metadata comprises information indicating an identity of a user (Col 2, lines 1-2 
"Bookmarks within a single program can be set for different users"). 

Regarding claim 12, Brown et al disclose an audiovisual system wherein at 
least one cell comprises a fixed image (Col 13, lines 65-67 "Any temporal elements 
[e.g., names, icons, location indicators] are drawn onto the screen over the video 
loops"). 

Regarding claim 13, Brown et al disclose an audiovisual system wherein at 
least one cell comprises a video image (Col 5, lines 39-40 "when the parser 401 finds a 
video event, it directs the stream to the video DMA engine 402"). 

Regarding claim 14, Brown et al disclose an audiovisual system which provides 
to a user information regarding the program segments of the stored portion of the 
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audiovisual data (Col 21 , lines 23-27 "the invention displays a program guide area 2801 
to the user which is a list of the programs that are currently airing, was aired, or is 
scheduled on live TV") comprising: 

• a storage device to store and play back a portion of the audiovisual data (Fig. 
1 Hard Disk 105); 

• a grid generator to configure for the user display the information regarding the 
program segments of the stored portion of the audiovisual data, the 
information derived from metadata corresponding to the program segments of 
the stored portion of the audiovisual data, the information provided to the user 
via the user display in grid format with a plurality of grid elements (Col 21 , 
lines 46-48 The invention displays the program guide information in two 
different modes: two column or three column"), each grid element 
representing a program segment of the stored portion of the audiovisual data 
(Col 21, lines 27-29 "The CPU 713 parses the program guide information and 
creates program schedule information"), each grid element independently 
selectable in response to a user input (Col 15, lines 5-8 "When the user 
presses on the right side of the button 1403 on the remote control 1401, a 
more detailed program information screen is displayed to the user"); and 

• a program selector to select a grid element that represents a selected 
program segment of the stored portion of the audiovisual data in response to 
user input, the selected program segment selected based on the information 
derived from the metadata, whereby the audiovisual system selectively plays 
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back selected programs of the stored portion of the audiovisual data starting 
from selected program locations, thereby enabling a user to jump to and play 
back selected program segments of the plurality of programs (Col 15, lines 
44-45 "The user can, at any time, access any of his bookmarks and continue 
viewing the program from the bookmark"). 

Brown et al show a grid of either rows or columns, but do not specifically 
disclose a grid of both a plurality of rows and a plurality of columns. 

Newman et al teach an audiovisual system with a grid of selectable 
elements corresponding to the program segments of the stored portion of the 
audiovisual data with a plurality of both rows and columns (Fig. 10, items 406). 

As taught by Newman et al, a grid displaying a plurality of both rows and 
columns of representative images from selectable video segments is well known, 
providing the user with a wide choice of selections of elements, and simplifying 
the user's decision making process regarding which element to select. 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention to modify Brown et al in order to include a grid of images 
with a plurality of both rows and columns. 

Regarding claim 16, Brown et al disclose an audiovisual system wherein at 
least one grid element comprises an icon which is representative of the program 
segment represented by the grid element (Fig. 24). 
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Regarding claim 17, Brown et al disclose an audiovisual system wherein at 
least one grid element comprises text which is representative of the program segment 
represented by the grid element (Fig. 24). 

Regarding claim 18, Brown et al disclose an audiovisual system wherein the 
plurality of grid elements are chosen for display based on the metadata of the 
corresponding program segments of the stored portion of the audiovisual data (Col 23, 
lines 20-22 "The program guide information is sorted in different ways, e.g., all movies 
2901, favorite channels 3001, all channels 3101, all sports, etc."). 

Regarding claims 19 and 21, Brown et al disclose an audiovisual system 
wherein the plurality of grid elements comprises grid elements chosen for display based 
on a second and third user input (Col 16, lines 12-14 "Specific remote controls] can be 
set up so that different levels of parental controls are engaged for each remote control"). 

Regarding claim 20, Brown et al disclose an audiovisual system wherein the 
plurality of grid elements are organized within the grid format in response to the 
metadata (Col 23, lines 20-22 "The program guide information is sorted in different 
ways, e.g., all movies 2901, favorite channels 3001, all channels 3101, all sports, etc."). 

Regarding claim 22, Brown et al disclose an audiovisual system wherein the 
plurality of grid elements are organized within the grid format based on the program 
from which each program segment is derived (Col 23, lines 20-22 The program guide 
information is sorted in different ways, e.g., all movies 2901, favorite channels 3001, all 
channels 3101, all sports, etc."). 
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Regarding claim 23, Brown et al disclose a method of playing back selected 
portions of stored audiovisual data stored on a storage device comprising: 

• providing stored audiovisual data corresponding to a plurality of programs, 
each program comprising a plurality of program locations, each program 
location representing a starting point of a program segment of one of the 
plurality of programs (Col 16, lines 51-60 "The user can rotate through three 
different levels of banners, each successively containing more information 
about the program. The lowest level banner 2001 contains minimal 
information such as channel 2002, station ID 2003, and time 2004. The 
second level banner 2005 displays, in addition to the information in the 
minimal banner 2001, information such as program title 2006, duration 2007, 
program MPAA or TV rating 2008, and thumbs rating. The final level banner 
2009 adds program text description 2010 to the second level banner 2005"); 

• creating metadata in response to a control input for marking the program 
segments comprising information regarding the program segments of the 
stored audiovisual data (Col 15, lines 35-38 "a user can watch the first half- 
hour of a two-hour program stored in the storage device 710 and then 
bookmark the program where he left off or he can place bookmarks within the 
program to mark points of interest" and lines 53-54 "A bookmark is placed in 
the same manner by pressing the select button 1406 to create a bookmark"); 
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• displaying to a user the information regarding the program segments of the 
stored audiovisual data (Col 15, lines 42-44 "The invention places an indicator 
1808 on the screen, indicating that a bookmark has been saved"); 

• receiving a user input indicating a selected program segment of one of the 
plurality of programs based on the information of the metadata (Col 15, lines 
50-52 "the user can jump to a bookmark using the jump button 1414 on the 
remote control 1401" and Col 15, lines 44-45 "The user can, at any time, 
access any of his bookmarks and continue viewing the program from the 
bookmark"); and 

• playing back the selected program segment starting from the corresponding 
program location of the stored audiovisual data, thereby jumping to and 
playing back selected program segments of the plurality of programs based 
on the metadata (Col 15, lines 44-45 "The user can, at any time, access any 
of his bookmarks and continue viewing the program from the bookmark"). 

Brown discloses images providing the user with additional data regarding 
the recorded program segments of a program (Col 16, lines 53-62), but does not 
explicitly disclose those images as being extracted from the respective stored 
program segments. 

Newman teaches an audiovisual recording and reproducing system 
comprising a menu of program segments comprising images extracted from the 
respective stored program segments (Col 15, lines 4-8 "the storyboard GUI 440 
includes a shot tab 404 displaying icons 406 representing previously captured 
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clips. The storyboard GUI 440 likewise includes a.display window 408 to display 

an image from a clip referenced by a selected icon 410"). 

As taught by Newman, a representation of a video program segment using 

a representative image extracted from the video program segment is well known, 

providing the user with a visual reference to the visual elements of the segment 

and simplifying the user's process of deciding which segment to view. 

Therefore, it would have been obvious to one of ordinary skill in the art at 

the time of the invention to modify Brown in order to include a representative 

image extracted from the video segment in an index of those video segments. 

Regarding claim 24, please refer to Examiner's comments regarding claim 4. 

Regarding claim 25, please refer to Examiner's comments regarding claim 7. 

Regarding claim 28, please refer to Examiner's comments regarding claim 12. 

Regarding claim 27, please refer to Examiner's comments regarding claim 13. 
4. Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination as applied to claims above, and further in view of Yuen (6,240,241 ) 

Regarding claim 15, Brown et ai disclose an audiovisual system wherein at 
least one grid element comprises an image (Figs 22 and 24), but do not specifically 
disclose that the image is extracted from the program segment represented by the grid 
element. 

Yuen teaches using representative images selected from the program in an index 
to help the user identify the program he is selecting (Col 1 , lines 46-48 "A frame of the 
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video program that is recorded on tape is selected to be displayed with the directory in 
text form for one or more of the recorded programs"). 

. As suggested by Brown and taught by Yuen, using an image from a file in an 
index is well known and commercially available, providing the user with a more useful 
index entry. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Brown in order to provide a representative image icon in 
the index grid. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James A. Fletcher whose telephone number is (571 ) 
272-7377. The examiner can normally be reached on 7:45-5:45 M-Th, first Fridays off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, James Groody can be reached on (571) 272-7950. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




JAF 

2 March 2007 



JamesJ.Greofly 
Superasoiyfttent Examiner 
Art Unit 26? 



